Location-Based Responses to Telephone Requests

ABSTRACT

A method for receiving processed information at a remote device is described. The method includes transmitting from the remote device a verbal request to a first information provider and receiving a digital message from the first information provider in response to the transmitted verbal request. The digital message includes a symbolic representation indicator associated with a symbolic representation of the verbal request and data used to control an application. The method also includes transmitting, using the application, the symbolic representation indicator to a second information provider for generating results to be displayed on the remote device.

TECHNICAL FIELD

Implementations are described that relate to information retrieval, andmore particularly to information retrieval based on verbal requests.

BACKGROUND

As computers and computer networks become more and more able to access awide variety of information, people are demanding more ways to obtainthat information. Specifically, people now expect to have access, on theroad, in the home, or in the office, to information previously availableonly from a permanently-connected personal computer hooked to anappropriately provisioned network. They want graphical maps of locationsfrom their cell phones, access to calendars from their personal digitalassistants (PDAs), up-to-date contact information from their emaildevices, and timely, accurate search results from all their devices.They also want all of this information when traveling, whether locally,domestically, or internationally, in an easy-to-use, portable device.

Some systems provide information to a user when the user calls from acell phone. For example, a user interested in the address for aparticular business can call a directory service, where a customerservice representative can provide the user with the business's address.This information, however, may not be provided in graphical form, forexample, as a map. Additionally, other useful information, such asdirections to the location and other similar businesses nearby may notbe provided by the customer service representative. Also, the user mayhave to wait until a customer service representative is available totake the user's call.

In other systems, a user may enter a request for information, such as amap of a business location, by inputting the request using a keypad of amobile device. Portability, however, generally requires a device smallin size, which in turn limits the number of data entry keys and theamount of memory and available processing power. In addition, ultraportable devices often must be held in one hand or not held at all, sothat data entry must be one-handed or no-handed. This can make enteringinformation to obtain rich content, such as graphical maps, difficult.

SUMMARY

Implementations are described that relate to information retrieval basedon verbal requests.

In a first general aspect, a method for receiving processed informationat a remote device is described. The method includes transmitting fromthe remote device a verbal request to a first information provider andreceiving a digital message from the first information provider inresponse to the transmitted verbal request. The digital message includesa symbolic representation indicator associated with a symbolicrepresentation of the verbal request and data used to control anapplication. The method also includes transmitting, using theapplication, the symbolic representation indicator to a secondinformation provider for generating results to be displayed on theremote device.

In a second general aspect, a method for providing a user withlocation-based information is described. The method includes receiving averbal request for locational information from a mobile device andconverting the verbal request into a symbolic representation. The methodalso includes generating a message including the symbolic representationand an indicator used to control an application on the mobile device.Additionally, the method includes transmitting the message to the mobiledevice to cause the application to use the symbolic representation todisplay one or more locations associated with the symbolicrepresentation.

In another general aspect, a method for receiving location-basedinformation is described. The method includes transmitting from a mobiledevice a verbal request for locational information to a firstinformation provider and receiving a Short Message Service message fromthe first information provider including a symbolic representation ofthe verbal request and an identifier used to control an application onthe mobile device. The method also includes transmitting a data requestincluding the symbolic representation of the verbal request forlocational information to a second information provider using theapplication on the mobile device and displaying on the mobile device oneor more locations associated with the symbolic representation usinglocational information transmitted by the second information provider inresponse to the data request.

In yet another general aspect, a method for providing processed data toa remote device is described. The method includes receiving a verbalrequest from the remote device and generating a data response for theverbal request. The data response includes data for controlling anapplication on the remote device and a conversion of the verbal request,wherein the converted verbal request is accessible by the application.The method also includes transmitting the data response to the remotedevice to cause the remote device to control the application, process atleast a portion of the converted verbal request, and display resultsgenerated from the processing.

In another general aspect, a system for providing location-basedinformation to a plurality of users is described. The system includes aninterface for receiving verbal requests over a voice transmission linefrom a plurality of remote devices, a conversion module for convertingthe verbal requests into a symbolic representation of the verbalrequest, and means for outputting digital messages to the remotedevices. Each message includes an identifier used to control anapplication on a remote device and a symbolic representation indicatorused by the application to process the symbolic representation.

The systems and techniques described here may provide one or more of thefollowing advantages. First, a system may increase the amount and typeof information accessible to a remote device by passing, to anapplication on the remote device, seed information used to deriveadditional information. Second, a system decreases inconvenience of textentry on mobile devices. Third, a system provides increased efficiencyand accuracy by offloading voice-recognition functions from a mobiledevice to a server. Fourth, a system increases efficiency because itenables transportation protocols of limited size be used to provide richcontent to mobile devices.

The details of one or more implementations of the invention are setforth in the accompanying drawings and the description below. Otherfeatures, objects, and advantages of the invention will be apparent fromthe description and drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram depicting an example of a system forprocessing a verbal request to present information on a remote device.

FIG. 2 is a schematic diagram of a system 200 depicting an automatedimplementation of the system 100 of FIG. 1.

FIG. 3 is a schematic diagram of a system 300 depicting a partiallymanual implementation of the system of FIG. 1.

FIG. 4 is a sequence diagram depicting an example of interactionsbetween a directory service provider, a cellular phone, and a searchengine.

FIG. 5 is a flow chart depicting an example of a method for displayingon a remote device information derived from a verbal request.

FIG. 6 is a schematic diagram depicting an example of a general computersystem and a general mobile device that may be used in the systems andmethods described in FIGS. 1-5.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

System 100 can translate a verbal request 102 from a remote device, suchas a voice telephone call from a cell phone, into a symbolicrepresentation, such as text, which can be used by an application on theremote device to initiate a query for information, for example, a searchquery.

FIG. 1 is a schematic diagram depicting an example of the system 100 forprocessing the verbal request 102 to present information on the remotedevice 104. The system 100 includes a first information provider 106 anda second information provider 108. In certain implementations, the firstinformation provider 106 is a directory assistance service, such as a“411” service, and the verbal request 102 is a call to the directoryassistance service. In these and other implementations, the secondinformation provider may be a search engine that processes searchqueries and returns search results based on the queries.

The remote device 104 (e.g., a cellular phone, a personal digitalassistant, or an e-mail device) transmits the verbal request 102 to thefirst information provider 106. The verbal request 102 may include, forexample, a request for a list of restaurants of a particular type in aparticular location (e.g., “Pizza, Palo Alto”).

The first information provider 106 receives the verbal request 102 andconverts the verbal request 102 into a symbolic representation. Forexample, the first information provider 106 may perform a speech-to-textoperation on the verbal request 102, which converts the verbal requestto ASCII (“American Standard Code for Information Interchange” text. Inother examples, the first information provider 106 can convert theverbal request into other symbolic representations, such as binary orhexadecimal, which the remote device 104 can use in a query 114described below.

In other implementations, the first information provider 106 can convertthe verbal request to character sets other than ASCII. For example, inan international context, the verbal request can be converted into150-8858-1 or UTF-8 (8-bit Unicode Transformation Format).

The first information provider 106 transmits a symbolic representationindicator 110 to the remote device 104 that indicates what the symbolicrepresentation is or from where the symbolic representation may beretrieved. For example, the symbolic representation indicator can besubstantially the same as the converted symbolic representation, or thesymbolic representation indicator can be a pointer or address location(e.g., uniform resource locator) of the converted symbolicrepresentation.

Additionally, the first information provider 106 transmits anapplication identifier 112 to the remote device 104. The remote device104 uses the application identifier to launch an application, such as amapping and directions application. The remote device 104 uses theapplication to transmit the symbolic representation indicator to thesecond information provider 108. For example, the remote device 104 maysend the query 114 to the second information provider 108, including thesymbolic representation indicator 110, requesting a map of restaurantsin the Palo Alto area that serve pizza.

The second information provider 108 processes the query 114 andtransmits results 116 of processing the query 114 to the remote device104. For example, the second information provider 108 may transmit a mapto the remote device 104 that highlights restaurants in Palo Alto thatserve pizza. The remote device 104 receives the results 116 and presentsdisplayed results 118, such as the map of restaurants, to a user.

Combining the examples above, a cell phone 104 can transmit a “411”request for “Pizza, Palo Alto,” as shown by arrow A. The request isreceived by a directory assistance provider 106, which converts theverbal request to text 110 and transmits the text 110 along with abinary message used to launch a mapping program on the remote device104, as shown by arrow B. The remote device 104 receives the binarymessage, launches the mapping program by executing the binary message,and uses the mapping program to transmit to a mapping server 108 a query“Pizza, Palo Alto” 118, as shown by arrow C. The mapping server 108processes the query and returns results 116 including a map highlightingpizza restaurants in Palo Alto, Calif., as shown by arrow D. Then remotedevice displays the map to the user.

FIG. 2 is a schematic diagram of a system 200 depicting an automatedimplementation of the system 100 of FIG. 1. The remote device 104, thefirst information provider 106, and the second information provider 108are in communication via a network 202, such as a local area network, awide area network, the Internet, a telephone network, or a combinationof networks.

The first information provider 106 receives the verbal request 102 viaan interface 204. In certain implementations, the remote device 104transmits a recorded message as the verbal request 102. For example, auser may record the request before connection to the first informationprovider. In other implementations, the verbal request is made after anestablished connection with the first service provider 106.

The first information provider 106 includes a speech-to-text converter206 that converts the verbal request 102 into text. An applicationselector 208 selects the application to be launched at the remote device104 based on, for example, the content of the converted verbal request.For example, the application selector 208 may determine from thelocation name, “Palo Alto,” that the user is requesting a map oflocations within Palo Alto. The application selector 208 generates theapplication identifier 112 based on the selected application. Theapplication identifier 112, when received at the remote device 104,instructs the remote device 104 to launch the selected application.

In other implementations, the application selector is optional becausethe application identifier 112 is predetermined so that every verbalrequest is related to a single application. For example, every requestmade to the first service provider may be for mapping locations, so theapplication identifier 112 may be binary code that always launches amapping application.

A message generator 210 uses the converted verbal request to generate adigital message 212 for transmission to the remote device. In certainimplementations, the digital message 212 includes the applicationidentifier 112 and the symbolic representation indicator. The firstinformation provider 106 transmits the digital message 212 to the remotedevice 104. In certain implementations, the message generator 210 uses aShort Message Service (SMS) formatter 214 to format the digital message212 as an SMS message. The application identifier 112 and/or thesymbolic representation indicator 110 may include binary values orAmerican Standard Code for Information Interchange (ASCII) values.

In certain implementations, the first information provider 106 includesa search engine 216 capable of performing a search using the symbolicrepresentation of the verbal request 102. The search engine 216 outputsthe search results to the message generator 210. The message generator210 includes partial results 218 for the verbal request 102 in thedigital message 212. For example, the partial results 218 may include inthe SMS message an address for a pizza parlor, which was returned as thetop result for the verbal request “Pizza, Palo Alto.” In anotherexample, the partial results 281 include a list of the top 3, 5, or 10results. A user may make a selection from the partial results 218 to beused in the query 114 to the second information provider 108, such as aparticular pizza restaurant in Palo Alto to highlight on a map insteadof all pizza restaurants in Palo Alto.

The remote device 104 uses the application identifier 112 to launch anapplication 220. In certain implementations, the application 220 is acalendar application and the verbal request 102 is a request to manageevent information in a calendar. In other implementations, theapplication 220 is a mapping application and the verbal request 102 is arequest to display one or more locations on a map. The locations may beidentified, for example, by keywords (e.g., pizza) and/or addresses(e.g., Palo Alto). In still other implementations, the application 220is a contact application and the verbal request 102 is a request tomanage contact information. In other implementations, the application220 is a search result user interface (e.g., web browser accessing aninternet search page) and the verbal request 102 is a search query.

After launching, the application 220 formats the symbolic representationfor inclusion in the query 114. The remote device 104 transmits thequery 114, including the symbolic representation indicator 110, to thesecond information provider 108.

The second information provider 108 includes a symbolic representationprocessor 222 that processes the symbolic representation of the verbalrequest 102 received in the query 114. The symbolic representationprocessor 222 includes a search engine 224 that performs a search usingthe query 114, such as a search of pizza restaurants.

In certain implementations, the first information provider 106 places asymbolic representation 226 of the verbal request 102 in a storagelocation 228 and the symbolic representation indicator 110 indicates thelocation of the symbolic representation 226, such as with a UniformResource Locator (URI). For example, where the symbolic representation226 of the verbal request is too large to be included in an SMS message,the symbolic representation 226 may be placed in the storage location228. The second information provider 108 retrieves the symbolicrepresentation 226 using the symbolic representation indicator 110 andprocesses the query 114 using the symbolic representation 226.

The remote device 104 receives the results 116 from the secondinformation provider 108 and presents the displayed results 118 to auser. In certain implementations, results including map locations mayalso include directions to the locations. To calculate the directions,the starting location may be input by a user of the remote device 104,or the starting location may be provided by a global positioningcomponent of the remote device 104. Additionally, a user may enter inputwhich causes a displayed map to show more or less detail of the map,such as by zooming in or out on the map.

FIG. 3 is a schematic diagram of a system 300 depicting a partiallymanual implementation of the system of FIG. 1. In this example, a person302 at the first information provider 104 receives the verbal request102. Here, the interface 204 includes a telephone interface. The person302 translates the verbal request 102 and inputs the symbolicrepresentation indicator 110 into a computer device 304 using an inputdevice 306, such as a keyboard, pointing device, or touch screen. Inthis example, the person 302 performs the operations of thespeech-to-text converter 206. Additionally, the person 302 can input theapplication identifier 112 and/or the partial results 218 for the verbalrequest 102 performing the operations of the application selector 208and the search engine 216, respectively.

FIG. 4 is a sequence diagram depicting an example of interactions 400between a directory service provider 402, a cellular phone 404, and asearch engine 406. The cellular phone 404 transmits a verbal request 408to the directory service provider, such as “Pizza, Palo Alto.”

The directory service provider 402 receives the verbal request 408 and,using a speech-to-text conversion, converts the verbal request 408 intoa symbolic representation. In certain implementations, the directoryservice provider 402 transmits a confirmation request 410 to thecellular phone 404. The confirmation request 410 can includecomputer-generated speech based on the symbolic representation, such as“Did you say, ‘Pizza, Palo Alto?” In other implementations, thedirectory service provider 402 may transmit a confirmation request 410including a text query containing the symbolic representation. Forexample, the confirmation request 410 may be an SMS message with thetext “Did you say, ‘Pizza, Palo Alto?” The cellular phone 404 transmitsa confirmation response 412 to the directory service provider 402 fromthe user, such as “Yes, I did.” The confirmation response 412 may beverbal or symbolic, such pressing the key ‘1’ on a keypad to indicate“Yes” and pressing the key ‘2’ to indicate “No.”

In other implementations, the user may correct the translation of theverbal request by verbally repeating the request, by typing the textusing a keypad, or by selecting from a list of possible translations ofthe verbal request.

The directory service provider 402 generates a binary SMS message 414and transmits the message 414 to the cellular phone 404. The message 414includes text, such as “Pizza, Palo Alto,” and an applicationidentifier, such as binary code used to launch or execute a function ofan internet browser or a custom application (e.g., Google Maps forMobile developed by Google™ of Mountain View, Calif.) installed on thecellular phone 404.

The cellular phone 404 receives the message 414 and launches the browser416 (or the custom application) with a map user interface (UI) based onthe included application identifier. The browser 416 with map userinterface transmits a data request 418 to the search engine 406including the symbolic representation, such as “Pizza, Palo Alto.”

The search engine 406 generates location search results 420 using thesymbolic representation in the data request 418 (or from the storagelocation 228 as previously described). For example, the search engine406 may determine that “Pizza” indicates a request for pizza restaurantsand “Palo Alto” further indicates a request for pizza restaurantlocations in the Palo Alto area. The search engine 406 transmits searchresults 422 to the cellular phone 404. The cellular phone 404 receivesthe results 422 and displays locations 424 using the map user interface.

FIG. 5 is a flow chart depicting an example of a method for displayingon a remote device information derived from a verbal request. Theprocess 500 may be performed, for example, by a system such as thesystems 100, 200, and 400. For clarity of presentation, the descriptionthat follows uses the systems 100, 200, and 400 as the basis of anexample for describing the process 500. However, another system, orcombination of systems, may be used to perform the process 500.

The process 500 begins with transmitting (502) a verbal request to afirst information provider. For example, the remote device 104 transmitsthe verbal request 102, including “Pizza, Palo Alto,” to the firstinformation provider 106 via the network 202.

Optionally, a confirmation request is received (504) from the firstinformation provider. The confirmation request seeks to confirm that aconversion of the verbal request to a symbolic representation, such asspeech-to-text, is accurate. For example, the cellular phone 404receives, from the directory service provider 402, the confirmationrequest 410 “Did you say, ‘Pizza, Palo Alto?”

Optionally, a confirmation of the verbal to symbolic conversion istransmitted (506). For example, a user replies “Yes, I did,” which istransmitted by the cellular phone 404 to the directory service provider402.

An application identifier and a symbolic representation indicator isreceived (508). For example, the remoter device 104 receives the digitalmessage 212 including the application identifier 112 and the symbolicrepresentation indicator 110.

An application specified by the application identifier is launched(510). For example, the remote device 104 launches the application 220specified by the application identifier 112.

If partial results are received in response to the verbal request 512,then the results an be displayed (514). If a user requests more resultsor the partial results were not received (516), then the process 500transmits (518) the symbolic representation indicator to a secondinformation provider.

For example, the remote device 104 may present the partial results 218including addresses for locations associated with the verbal request andreceive a user input requesting more results. If a user requests moreresults or partial results were not transmitted, the remote device 104transmits the symbolic representation indicator 110, including “Pizza,Palo Alto,” to a search engine that returns mapped locations in responseto location queries.

Processed results are received from the second information provider(520). For example, the remote device 104 receives the results 116 fromthe second information provider 108, including a map with locations ofpizza restaurants in Palo Alto highlighted.

FIG. 6 is a block diagram of computing devices 600, 650 that may be usedto implement the systems and methods described in this document, eitheras a client, as a server, or as a plurality of servers. Computing device600 is intended to represent various forms of digital computers, such aslaptops, desktops, workstations, personal digital assistants, servers,blade servers, mainframes, and other appropriate computers. Computingdevice 650 is intended to represent various forms of mobile devices,such as personal digital assistants, cellular telephones, smart phones,and other similar computing devices. The components shown here, theirconnections and relationships, and their functions, are meant to beexemplary only, and are not meant to limit implementations of theinventions described and/or claimed in this document.

Computing device 600 includes a processor 602, memory 604, a storagedevice 606, a high-speed interface 608 connecting to memory 604 andhigh-speed expansion ports 610, and a low speed interface 612 connectingto low speed bus 614 and storage device 606. Each of the components 602,604, 606, 608, 610, and 612, are interconnected using various busses,and may be mounted on a common motherboard or in other manners asappropriate. The processor 602 can process instructions for executionwithin the computing device 600, including instructions stored in thememory 604 or on the storage device 606 to display graphical informationfor a GUI on an external input/output device, such as display 616coupled to high speed interface 608. In other implementations, multipleprocessors and/or multiple buses may be used, as appropriate, along withmultiple memories and types of memory. Also, multiple computing devices600 may be connected, with each device providing portions of thenecessary operations (e.g., as a server bank, a group of blade servers,or a multi-processor system).

The memory 604 stores information within the computing device 600. Inone implementation, the memory 604 is a computer-readable medium. In oneimplementation, the memory 604 is a volatile memory unit or units. Inanother implementation, the memory 604 is a non-volatile memory unit orunits.

The storage device 606 is capable of providing mass storage for thecomputing device 600. In one implementation, the storage device 606 is acomputer-readable medium. In various different implementations, thestorage device 606 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device, a flash memory or other similarsolid-state memory device, or an array of devices, including devices ina storage area network or other configurations. In one implementation, acomputer program product is tangibly embodied in an information carrier.The computer program product contains instructions that, when executed,perform one or more methods, such as those described above. Theinformation carrier is a computer- or machine-readable medium, such asthe memory 604, the storage device 606, memory on processor 602, or apropagated signal.

The high speed controller 608 manages bandwidth-intensive operations forthe computing device 600, while the low speed controller 612 manageslower bandwidth-intensive operations. Such allocation of duties isexemplary only. In one implementation, the high-speed controller 608 iscoupled to memory 604, display 616 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 610, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 612 is coupled to storage device 606 and low-speed expansionport 614. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 600 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 620, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 624. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 622. Alternatively, components from computing device 600 may becombined with other components in a mobile device (not shown), such asdevice 650. Each of such devices may contain one or more of computingdevice 600, 650, and an entire system may be made up of multiplecomputing devices 600, 650 communicating with each other.

Computing device 650 includes a processor 652, memory 664, aninput/output device such as a display 654, a communication interface666, and a transceiver 668, among other components. The device 650 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 650, 652,664, 654, 666, and 668, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 652 can process instructions for execution within thecomputing device 650, including instructions stored in the memory 664.The processor may also include separate analog and digital processors.The processor may provide, for example, for coordination of the othercomponents of the device 650, such as control of user interfaces,applications run by device 650, and wireless communication by device650.

Processor 652 may communicate with a user through control interface 658and display interface 656 coupled to a display 654. The display 654 maybe, for example, a TFT LCD display or an OLED display, or otherappropriate display technology. The display interface 656 may compriseappropriate circuitry for driving the display 654 to present graphicaland other information to a user. The control interface 658 may receivecommands from a user and convert them for submission to the processor652. In addition, an external interface 662 may be provide incommunication with processor 652, so as to enable near areacommunication of device 650 with other devices. External interface 662may provide, for example, for wired communication (e.g., via a dockingprocedure) or for wireless communication (e.g., via Bluetooth or othersuch technologies).

The memory 664 stores information within the computing device 650. Inone implementation, the memory 664 is a computer-readable medium. In oneimplementation, the memory 664 is a volatile memory unit or units. Inanother implementation, the memory 664 is a non-volatile memory unit orunits. Expansion memory 674 may also be provided and connected to device650 through expansion interface 672, which may include, for example, aSIMM card interface. Such expansion memory 674 may provide extra storagespace for device 650, or may also store applications or otherinformation for device 650. Specifically, expansion memory 674 mayinclude instructions to carry out or supplement the processes describedabove, and may include secure information also. Thus, for example,expansion memory 674 may be provide as a security module for device 650,and may be programmed with instructions that permit secure use of device650. In addition, secure applications may be provided via the SIMMcards, along with additional information, such as placing identifyinginformation on the SIMM card in a non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, asdiscussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 664, expansionmemory 674, memory on processor 652, or a propagated signal.

Device 650 may communicate wirelessly through communication interface666, which may include digital signal processing circuitry wherenecessary. Communication interface 666 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 668. In addition, short-range communication may occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, GPS receiver module 670 may provide additional wireless datato device 650, which may be used as appropriate by applications runningon device 650.

Device 650 may also communication audibly using audio codec 660, whichmay receive spoken information from a user and convert it to usabledigital information. Audio codex 660 may likewise generate audible soundfor a user, such as through a speaker, e.g., in a handset of device 650.Such sound may include sound from voice telephone calls, may includerecorded sound (e.g., voice messages, music files, etc.) and may alsoinclude sound generated by applications operating on device 650.

The computing device 650 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 680. It may also be implemented as part of a smartphone 682, personal digital assistant, or other similar mobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of embodiments of the invention have been described.Nevertheless, it will be understood that various modifications may bemade without departing from the spirit and scope of the invention. Forexample, the first information provider 106 and the second informationprovider 108 may be the same system or may be included in a singlesystem.

Additionally, the application identifier 112 does not have to launch theapplication. Instead, it can be used to initiate a function of anapplication that has been previously launched. For example, theapplication identifier can indicate that the a previously launchedinternet browser should navigate to a particular site identified by aURL included in the digital message 212.

In some implementations, the request 102 is a written request and theremote device 104 that transmits the request 102 is a personal computer,and the request 102 can be passed directly to the search engine 216 andthe application selector without speech-to-text conversion.

Although SMS is used as an example in the above descriptions, in otherimplementations, the described systems and method may use a variety ofmechanisms to transmit information to the remote device 104. Forexample, the first information provider 106 can use actual pushmechanisms, such as session-initiated protocol (SIP) and SMStransmissions, or virtual push mechanisms, such as polling usinghypertext transmission protocol (HTTP).

Additionally, in some implementations, the request 102 can split up intomultiple requests. For example, the request 102 can be a two-phaseinteraction, where a user of the remote device requests a location first(e.g., Dallas, Tex.), which is confirmed by the first informationprovider. Next, the user transmits a second portion of the request,which specifies a particular category or item for which to search (e.g.,barbeque).

In yet other implementations, various forms of the flows shown above maybe used, with steps re-ordered, added, or removed. Also, althoughseveral applications of the systems and methods have been described, itshould be recognized that numerous other applications are contemplated.Accordingly, other embodiments are within the scope of the followingclaims.

1. (canceled)
 2. A method comprising: receiving, at a first computersystem, audio data associated with a verbal request from a user, theuser associated with a computing device; converting, by the firstcomputer system, the received audio data into a symbolic representationof content of the verbal request; generating, by the first computersystem, an application identifier for a particular application to belaunched on the computing device, the particular application indicatedby the content of the verbal request; and transmitting the symbolicrepresentation and the application identifier to the computing device,the symbolic representation and the application identifier when receivedby the computing device causing the computing device to: launch theparticular application installed on the computing device; transmit aquery to a second computer system that corresponds to the particularapplication, the query based on the symbolic representation of theverbal request; receive a response to the query from the second computersystem; and present the response to the query through an interface ofthe computing device.
 3. The method of claim 2, wherein: receiving theaudio data associated with the verbal request comprises receiving arecording of the verbal request spoken by the user; and converting thereceived audio data into the symbolic representation comprises using aspeech-to-text converter to convert the recording of the verbal requestinto a textual representation of the verbal request.
 4. The method ofclaim 2, further comprising, prior to generating the applicationidentifier, identifying, by the first computer system, the particularapplication from a plurality of possible applications installed on thecomputing device as having significance to the content of verbal requestfrom the user.
 5. The method of claim 2, wherein the computing device isconfigured to: present the interface; and launch the particularapplication within the interface.
 6. The method of claim 2, wherein thecomputing device is configured to generate the query by formatting thequery to include data based on the symbolic representation of the verbalrequest.
 7. The method of claim 2, wherein, after the computing devicelaunches the particular application, the particular application isconfigured to format the symbolic representation for inclusion in thequery.
 8. The method of claim 2, wherein the computing device isconfigured to, after launching the particular application, use theapplication identifier to establish a connection with the secondcomputer system.
 9. The method of claim 2, wherein the symbolicrepresentation of content of the verbal request comprises a textualrepresentation of the verbal request.
 10. The method of claim 2, whereinthe computing device presents the response to the query through theinterface as an output of the launched particular application.
 11. Themethod of claim 2, wherein the particular application is installed onthe computing device.
 12. A system comprising: data processing hardwareof a first computer system; memory hardware of the first computer systemand in communication with the data processing hardware, the memoryhardware storing instructions that when executed by the data processinghardware cause the data processing hardware to perform operationscomprising: receiving audio data associated with a verbal request from auser, the user associated with a computing device; converting thereceived audio data into a symbolic representation of content of theverbal request; generating an application identifier for a particularapplication to be launched on the computing device, the particularapplication indicated by the content of the verbal request; andtransmitting the symbolic representation and the application identifierto the computing device, the symbolic representation and the applicationidentifier when received by the computing device causing the computingdevice to: launch the particular application installed on the computingdevice; transmit a query to a second computer system that corresponds tothe particular application, the query based on the symbolicrepresentation of the verbal request; receive a response to the queryfrom the second computer system; and present the response to the querythrough an interface of the computing device.
 13. The system of claim12, wherein: receiving the audio data associated with the verbal requestcomprises receiving a recording of the verbal request spoken by theuser; and converting the received audio data into the symbolicrepresentation comprises using a speech-to-text converter to convert therecording of the verbal request into a textual representation of theverbal request.
 14. The method of claim 12, wherein the operationsfurther comprise, prior to generating the application identifier,identifying the particular application from a plurality of possibleapplications installed on the computing device as having significance tothe content of verbal request from the user.
 15. The system of claim 12,wherein the computing device is configured to: present the interface;and launch the particular application within the interface.
 16. Thesystem of claim 12, wherein the computing device is configured togenerate the query by formatting the query to include data based on thesymbolic representation of the verbal request.
 17. The system of claim12, wherein, after the computing device launches the particularapplication, the particular application is configured to format thesymbolic representation for inclusion in the query.
 18. The system ofclaim 12, wherein the computing device is configured to, after launchingthe particular application, use the application identifier to establisha connection with the second computer system.
 19. The system of claim12, wherein the symbolic representation of content of the verbal requestcomprises a textual representation of the verbal request.
 20. The systemof claim 12, wherein the computing device presents the response to thequery through the interface as an output of the launched particularapplication.
 21. The system of claim 12, wherein the particularapplication is installed on the computing device.